This application is based upon and claims the benefit of priority from the prior Japanese Patent Applications No. 2000-283292, filed Sep. 19, 2000; and No. 2000-284318, filed Sep. 19, 2000, the entire contents of both of which are incorporated herein by reference.
1. Field of the Invention
The present invention relates to a three-dimensional position/orientation sensing apparatus for utilizing an image capturing apparatus to photograph an object and measuring a relative three-dimensional position/orientation of the object, an information presenting system, and a model error detecting system.
2. Description of the Related Art
A conventional technique has been known for analyzing an image obtained by capturing a marker as an index of a predetermined shape and obtaining a position and orientation of image input means with respect to the marker.
For example, the following technique is disclosed in xe2x80x9cHigh-Density Real-Time Estimating Method of Position/Orientation of Rectangular Marker by Monocule for VR Interfacexe2x80x9d (3D Image Conference 96 Drafts, pp. 167 to 172, Akira Takahashi, Ikuo Ishii, Hideo Makino, Makoto Nakano, 1996). In the technique, a rectangular marker whose position coordinate is known beforehand is captured, and a relative position/orientation relation between the rectangular marker and a capturing camera is obtained from positions of four corners of the rectangular marker on a captured image. Moreover, the position and orientation of the capturing camera are calculated.
However, when the camera is far from the marker in the aforementioned prior art, distances among the four corners of the marker on the image are reduced, and detection precision is disadvantageously deteriorated. Therefore, a detectable range is limited.
To avoid this, a plurality of markers are arranged beforehand at a certain degree of density in a space region which is possibly captured by the camera, and the marker having a relatively short distance may be used out of the markers in a camera capturing range to obtain the relative position/orientation of the camera and marker.
However, when the plurality of markers are used in this manner, and when a simple diagram is used as in the aforementioned prior art, it is difficult to designate the marker as a reference of the relative position.
On the other hand, various techniques of superimposing and displaying information of an object in the image onto the image captured by the camera have heretofore been developed.
For example, in Jpn. Pat. Appln. KOKAI Publication No. 267671/1998, a technique having the following characteristic is disclosed. That is, a view image is taken by the camera in the technique. Subsequently, a visual field space in a map information space is obtained based on a camera position and view angle, focal length, and image size information at the time. Moreover, a structure in the visual field space is acquired. Furthermore, a name and attribute of the structure are prepared as label information. Additionally, the label information is superimposed/displayed on the view image.
Moreover, a technique having the following characteristic is disclosed in Jpn. Pat. Appln. KOKAI Publication No. 56624/1995. That is, the technique is a system for use in maintenance and repair in an operation site. In a head mounted display, an image obtained by two-dimensionally projected information of a subject present around an operator, and an actual image masked so that only a subject portion is displayed are superimposed/displayed. An output of a gyro included in the head mounted display is used in controlling the mask.
However, in these prior arts, it is necessary to take in the position and view angle of capturing means, focal length, image size information, and the like. Therefore, exact superimposition is impossible in consideration of sensor error and drift. It is also necessary to measure a capturing position so that the position is accurate to some degree, and there is a limitation. Moreover, the name, attribute, and the like are displayed on the real image, and this only assists understanding of the operator.
There is a demand for comparison/study of object data with the actual object on the image in industrial fields, particularly in a field of facilities construction and maintenance. However, the aforementioned prior arts do not satisfy this demand.
The present invention has been developed in consideration of the aforementioned problems, and an object thereof is to provide a three-dimensional position/orientation sensing apparatus. The apparatus can measure a relatively broad region, when an image obtained by capturing a marker having a known position/orientation relation with an object is analyzed, a relative position/orientation relation between the marker and capturing means is obtained, and a position and orientation of the object are obtained. Further object is to provide an information presenting system and model error detecting system in which a captured image of an actual object can easily be compared with object data.
To achieve the object, according to a first aspect of the present invention, there is provided a three-dimensional position/orientation sensing apparatus for measuring a relative three-dimensional position/orientation relation between an object and a capturing apparatus from an image captured by the capturing apparatus, comprising: an image input section which inputs the image obtained by capturing the object and a marker having a known three-dimensional position/orientation relation by the capturing apparatus; an identifying section which uses a geometric characteristic of the marker to identify the marker in the image inputted from the image input section; and a position/orientation detecting section by a single marker, which analyzes the image of one of a plurality of markers identified by the identifying section, obtains a relative position/orientation relation between the marker and the capturing apparatus, and obtains the three-dimensional position/orientation relation between the object and the capturing apparatus.
According to a second aspect of the present invention, there is provided an information presenting system comprising: an image input section which inputs an image obtained by capturing a marker whose three-dimensional position relation with an object is known by a capturing apparatus; a position/orientation detecting section which uses a position of the marker on the image to obtain a position/orientation relation between the object and the capturing apparatus; and a display section which displays three-dimensional model data of the object on the image with a position, a size, and a direction based on the position/orientation relation.
According to a third aspect of the present invention, there is provided a model error detecting system in which three-dimensional model data of an object can be corrected based on an image of the object, the system comprising: an image input section which inputs the image obtained by capturing the object by a capturing apparatus; a position/orientation detecting section which obtains a position/orientation relation between the object and the capturing apparatus; a holding section which holds the model data of the object; an error detecting section which compares the object model data extracted from the holding section with the image of the object based on the position/orientation relation, and detects an actual error with the model data of the object; and a correcting section which corrects the model data of the holding section based on a detection result of the error detecting section.
Additional objects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objects and advantages of the invention may be realized and obtained by means of the instrumentalities and combinations particularly pointed out hereinafter.